Deploying radio frequency (RF) localisation systems invariably entails non-trivial effort, particularly for the latest learning-based breeds. There has been little prior work on characterising and comparing how learnt localiser networks can be deployed in the field under real-world RF distribution shifts. In this paper, we present RadioBench: a suite of 8 learnt localiser nets from the state-of-the-art to study and benchmark their real-world deployability, utilising five novel industry-grade datasets. We train 10k models to analyse the inner workings of these learnt localiser nets and uncover their differing behaviours across three performance axes: (i) learning, (ii) proneness to distribution shift, and (iii) localisation. We use insights gained from this analysis to recommend best practices for the deployability of learning-based RF localisation under practical constraints. Decades of of radio frequency (RF) localisation research have given us a variety of classic methods (Patwari et al., 2005; Gezici et al., 2005). Newer machine learning incarnations can enhance location estimation considerably (Zanjani et al., 2022; Karmanov et al., 2021), albeit at the expense of proneness to distributional shift in wireless signals. For example, models trained on signals from a warehouse environment may not work well in another different environment (Arnold et al., 2018). If learnt localiser networks are to be productised and deployed, it is imperative that we robustify them.